*******************************************************************************
*Replication script for: Höhne, J.K., Claassen, J., & Wolf, B.L. (in press).
*LLM-driven bot infiltration: Protecting web surveys through prompt injections.
*International Journal of Social Research Methodology.
*******************************************************************************

************
*Import data
************

use "prompt_injections.dta", clear


********************
*Experimental design
********************

*distribution of conditions
tab condition


*************************************************************
*Table 1. Prompt injection performance across LLM-driven bots
*************************************************************

*jailbreaking (positive)
tab bot positive_jailbreak, row

*jailbreaking (negative)
tab bot negative_jailbreak, row

*prompt leaking
tab bot prompt_leaking, row

*control (no injection)
tab bot control, row


*************************************************************
*Table SOM2. Response characteristics of the LLM and LLM+ bot
*************************************************************

*percentage of unique responses
tab bot unique_response, row

*response length (LLM bot)
sum response_length if bot == 1

*response length (LLM+ bot)
sum response_length if bot == 2

*percentage of meaningful responses
tab bot meaningful_response, row


*********************************************************************************
*Table SOM3. Prompt injection performance across persona settings of the LLM+ bot
*********************************************************************************

*jailbreaking (positive)
tab age_group positive_jailbreak, row

tab female positive_jailbreak, row

tab education positive_jailbreak, row

tab party positive_jailbreak, row

*jailbreaking (negative)
tab age_group negative_jailbreak, row

tab female negative_jailbreak, row

tab education negative_jailbreak, row

tab party negative_jailbreak, row

*prompt leaking
tab age_group prompt_leaking, row

tab female prompt_leaking, row

tab education prompt_leaking, row

tab party prompt_leaking, row

*Unique persona combinations
egen group_id = group(age_group female education party)

tab group_id /*200 LLM+ bot runs, with 77 unique persona combinations out of 108 possible combinations*/
